1. Field
Methods and apparatuses consistent with exemplary embodiments relate to estimating a light source in an image obtained by an image device.
2. Description of the Related Art
The human color perception system ensures that a perceived color of objects remains relatively constant under varying illumination conditions. For example, white paper looks white in sunlight. The white paper looks orange under a halogen lamp since the halogen lamp is an orange light source. However, humans perceive that the white paper looks white also under the halogen lamp, like in sunlight. A feature of the human color perception system which ensures that the human perceives the original color of objects under varying illumination conditions is referred to as color constancy. The color constancy refers to white balance adjustment in an electronic photographing device such as a digital still camera or a digital video camera. The white balance adjustment is an adjustment among RGB color channels so that an object with achromatic color may look achromatic. The white balance adjustment is impossible without information about spectral characteristics of an original light source and an object surface. That is, spectral characteristics detected by the human eye are expressed by a product of a spectral reflective characteristic of a light source and that of an object surface. Since these two characteristics are unknown, problems may arise with the white balance adjustment. Thus, various algorithms by which color constancy is modeled by making an assumption or limitation with respect to a light source, an object surface, and a viewed scene have been suggested.
One of algorithms that have been frequency used to address color constancy is gray hypotheses (for example, refer to JP 3767541). The gray hypotheses is based on experience data in that an average color of various colors is close to gray since various colors are distributed in a general scene within a view. If the average color deviates from gray, it is determined that the deviation is caused by color of a light source. Based on this determination, white balance may be adjusted by performing adjustment between RGB color channels. This method may have low calculating costs, and may be easily used in an internal device such as a digital still camera. On the other hand, since it is assumed that an object is physically dependent upon a light source, this assumption is not right according to a scene, and thus, an estimating result of a light source may vary. For example, when a gray logo exists on a portion of a red wall, an average color of an image is red. Here, according to the gray hypothesis, it is determined that the red color is caused by color of a light source. Thus, in order to make an entire scene gray, an opposite color of the red color is applied to the entire scene. Thus, the logo having an original gray color is changed to the opposite color of the red color. This is referred to as color feria.
A method of estimating a light source by using spectral characteristics of a light source and an object surface has been suggested (for example, refer to D. H. Brainard & W. T. Freeman, “Baysian Color Constancy”, J. Opt. Soc. Am. A, Vol. 14, No. 7, 1997). The method may statistically increase accuracy of estimating a light source by expressing the spectral characteristics of the light source and the object surface on an orthogonal basis with a low difference therebetween and making an assumption or limitation with respect to the light source, the object surface, and a photographing system, or combining information about the light source, the object surface, and the photographing system. When this method is used, the light source may be estimated in detail. However, since the number of calculating operations is increased, when the method is used to perform calculation for an image process such as a white balance process in a digital camera, it may be difficult to perform the process in real time.
As a method of estimating a type of light source in consideration with spectral characteristic of the light source and object surface while reducing calculating costs, a brightness-color correlation method has been suggested (for example, refer to J. Golz & D. I. A. Macleod, “Influence of scene statistics on color constancy”, NATURE, VOL. 415, 7, FEBRUARY, 2002). In this method, when a light source is red (which is the same as in cases where the light source is green or blue), it is determined whether the light source is white or not by using correlation in that a red color of a red region deepens. With reference to FIGS. 9 and 10, the brightness-color correlation will be described. FIGS. 9 and 10 are schematic diagrams of spectral distribution for showing brightness-color correlation. In FIGS. 9 and 10, a vertical axis denotes a wavelength, and a horizontal axis denotes energy corresponding to respective wavelength. FIG. 9A is a schematic diagram for showing, from a left side, spectral distribution of an image of a general object X including various colors, spectral distribution of a red light source (Lr) irradiating light to the object X, and spectral distribution of an image captured by irradiating light to the object X from the red light source Lr. In FIG. 9A, intensities of spectrums of images are reduced towards a short wavelength (i.e., blue) and are increased towards a long wavelength (i.e., red). That is, images that are captured by irradiating light from the red light source have a correlation in that a red degree of a region is increased as the region becomes brighter. This correlation is appropriate when a plurality of colors exist in an image. FIG. 9B is a schematic diagram for showing, from a left side, spectral distribution of an image of a red object Xr, spectral distribution of a white light source Lh (for example, a light source having a color temperature of 5,500 K) irradiating light to the object Xr, and spectral distribution of an image captured by irradiating light from the white light source Lh. Since the spectrum distribution of the white light source is flat, spectrum distribution of an object itself is detected. Thus, brightness and color has no correlation. In addition, this correlation may be applied to a blue degree as well as a red degree.
FIGS. 10A through 10C are schematic diagrams for showing spectrum distribution when light is irradiated from a red light source. FIG. 10A is a schematic diagram for showing, from a left side, spectral distribution of an image of a red region Sr, spectral distribution of a light source Lr irradiating light to the red region Sr, and spectral distribution of an image captured by irradiating light to the red region Sr from the red light Lr. FIG. 10B is a schematic diagram for showing, from a left side, spectral distribution of an image of a green region Sg, spectral distribution of a red light source Lr irradiating light to the green region Sg, and spectral distribution of an image captured by irradiating light to the green region Sg from the red light source Lr. FIG. 10C is a schematic diagram for showing, from a left side, spectral distribution of an image of a blue region Sb, spectral distribution of a red light source Lr irradiating light to the blue region Sb, and spectral distribution of an image captured by irradiating light to the blue region Sb from the red light source Lr. Since a long wavelength has low energy, when light is irradiated to the blue region Sb from the red light source Lr, the blue region Sb is darker than a case where light is irradiated to the red region Sr from the red light source Lr. Energy around a short wavelength of reflective light in the blue region Sb is reduced. However, the blue region Sb has more short wavelength components than a case where red light is irradiated to the red region Sr (this is because the blue region has a wide spectrum around a short wavelength). That is, there is negative correlation in that blue deepens as it becomes darker. So far, the “brightness-color correlation” has been described.
According to this method, overall characteristics of a light source (white or nonwhite) may be estimated by using a small number of calculating operations. This is because the detailed spectral characteristics do not have to be considered compared to the above-described method using spectral reflective characteristics. Thus, this method has a sufficient performance for overcoming problems with color feria and has low calculating costs. Thus, this method may be effectively used in an internal device.
In addition, JP 3767541 discloses a method of estimating spectral characteristics indicating colors of a light source irradiating light to an object from response values of sensors having different spectral sensitivity characteristics.
However, the above-described brightness-color correlation is affected by a degree of texture of an image, which is generated by a position of a camera or an intensity of a light source. For example, when the texture of the image is flat and the number of colors is small, since input data distribution of brightness and color is not spread out, the brightness-color correlation may be unstable. In a real environment, the influence that causes this problem needs to be reduced in order to stably perform estimation of a light source.
Since the brightness-color correlation is based on an assumption that input data distribution is normal distribution, when the number of input data is small or data has an outlier, the assumption is not appropriate, and thus, correct brightness-color correlation may not be calculated. In addition, some pixels look white due to specular reflection of an object. Thus, in a scene including a highlight region, the pixels correspond to outliers, and thus, the brightness-color correlation may be unstable.
In addition, in JP 3767541, the number of calculation operations is increased in order to estimating spectral characteristics indicating color of a light source based on a response value of a sensor. In addition, an imaging device needs to include sensors having different spectral sensitivity characteristics.